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(54) Improvements in or relating to image display systems 



(57) A video processing unit (13) that decodes com- 
pressed video data and resizes the image represented 
by the video data. The video processing unit (13) has 
two processing engines - a decoding engine (24) and a 
scaling engine (25). which share a memory (23). A 
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memory manager (22) handles data requests from the 
two engines, and handles reading and writing of the 
memory (22). 
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Description 

TECHNICAL FIELD OF THE INVENTION 

This invention relates to image display systems that 
use spatial light modulators (SLMs), and more particu- 
larly to an integrated circuit for decoding compressed 
input signals to generate images and for resizing the 
images to match the SLM display size. 



BACKGROUND OF THE INVENTION 

Video display systems based on spatial light modu- 
lators (SLMs) are increasingly being used as an alterna- 
tive to display systems using cathode ray tubes (CRTs). 
SLM systems provide high resolution displays without 
the bulk and power consumption of CRT systems. They 
are all-digital, with both digital processing and display. In 
other words, a digital input signal can be received, or an 
analog signal can be received and digitized, with subse- 
quent processing and display of the data in digital form. 

Digital micro-mirror devices (DMDs) are a type of 
SLM, that may be used for display applications. A DMD 
has an array of micro-mechanicai display elements, 
each having a tiny mirror that is individually addressable 
with an electronic signal. Depending on the state of its 
addressing signal, each mirror tilts so that it either does 
or does not reflect light to the image plane. The mirrors 
are often referred to as "pixels", to correspond to the 
pixels of the image they generate, but they are more 
correctly referred to as "display elements". Generally, 
displaying pixel data is accomplished by loading mem- 
ory cells connected to the display elements. Each mem- 
ory cell receives one bit of data representing an "ON " or 
"OFF" state of a display element. The display elements 
can maintain their "ON" or "OFF" state for controlled 
display times. 

Other SLMs operate on similar principles, with an 
array of display elements that may emit or reflect light 
simultaneously, such that a complete image is gener- 
ated by addressing display elements rather than by 
scanning a screen. Another example of an SLM is a liq- 
uid crystal display (LCD) having individually driven dis- 
play elements. 

For all types of SLMs, motion displays are achieved 
by updating the data in the SLMs memory cells at suffi- 
ciently fast rates. To achieve intermediate levels of illu- 
mination, between white (ON) and black (OFF), pulse- 
width modulation (PWM) techniques are used. The 
basic PWM scheme involves first determining the rate at 
which images are to be presented to the viewer. This 
establishes a frame rate and a corresponding frame 
period. For example, in a standard television system, 
images are transmitted at 30 frames per second, and 
each frame lasts for approximately 33.3 milliseconds. 
Then, the intensity resolution for each pixel is estab- 
lished. In a simple example, and assuming n bits of res- 
olution, the frame time is divided into 2 n -1 equal time 



slices. For a 33.3 millisecond frame period and n-bit 
intensity values, the time slice is 33.3/(2 n -1) millisec- 
onds. 

Having established these times, for each pixel of 
5 each frame, pixel intensities are quantized, such that 
black is 0 time slices, the intensity level represented by 
the LSB is 1 time slice, and maximum brightness is 2 n - 
1 time slices. Each pixel's quantized intensity deter- 
mines its ON-time during a frame period. Thus, during a 
jo frame period, each pixel with a quantized value of more 
than 0 is "ON" for the number of time slices that corre- 
spond to its intensity. The viewer's eye integrates the 
pixel brightness so that the image appears the same as 
if it were generated with analog levels of light. 
15 For addressing SLMs, PWM calls for the data to be 
formatted into "bit-planes", each bit-plane correspond- 
ing to a bit weight of the intensity value. Thus, if each 
pixel's intensity is represented by an n-bit value, each 
frame of data has n bit-planes. Each bit-plane has a 0 or 
20 1 value for each display element. In the PWM example 
described in the preceding paragraphs, during a frame, 
each bit-plane is separately loaded and the display ele- 
ments are addressed according to their associated bit- 
plane values. For example, the bit-plane representing 
25 the LSBs of each pixel is displayed for 1 time slice, 
whereas the bit-plane representing the MSBs is dis- 
played for 2 n /2 time slices. 

Digital display systems based on SLMs have a fixed 
image size -- the SLM is designed to display an image 
3C of a certain horizontal and vertical resolution. This fixed 
image size of the SLM does not necessarily match the 
image size of the input signal. If this is the case, the 
image must be resized to fit the SLM. This resizing has 
required special processing and memory, with a result- 
35 ant increase in the cost of the display system. For exam- 
ple, the processing algorithms of image resizers require 
special line buffers capable of delivering the same line 
more than once. 

SLM-based display system must also keep pace 
40 with other types of display systems with regard to the 
ability to accept input signals carrying compressed 
video data. Existing decompression processors are 
either general purpose processors having special 
decompression programming or are special purpose 
45 processors designed for use with a host processor. A 
need exists for a processor designed especially for use 
with an SLM-based system. 



SUMMARY OF THE INVENTION 
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One aspect of the invention is a digital video 
processing unit that receives an input signal carrying 
compressed video data. It decompresses the video data 
and resizes the image represented by the data. The 
55 processing unit is comprised of two processing 
"engines" -- a decoding engine for decompression and 
a scaling engine for resizing. These engines share a 
memory, which stores compressed video data for deliv- 
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ery to the decoding engine and also stores the video 
data after decompression for delivery to the scaling 
engine. A memory manager is in communication with 
the decoding engine, the scaling engine, and the mem- 
ory. The memory manager provides data as needed, 
responding to request signals from the decoding engine 
and the scaling engine. It also generates address sig- 
nals for storing data in the memory. 

An advantage of the invention is that sharing of 
memory by the decoding engine and scaling engine 
simplifies the memory requirements of the display sys- 
tem. Specifically, a special line buffer between the 
decoder and image resizer is not necessary as would 
be the case in the absence of a shared memory. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will now be further described, 
by way of example, with reference to the accompanying 
drawings in which: 

FIGURE 1 is a block diagram of a digital image dis- 
play system, which uses a spatial light modulator to 
generate full-color full-motion displays, and which 
has a video processing unit in accordance with the 
invention; 

FIGURE 2 is a block diagram of the digital video 
processing unit of FIGURE 1; and 
FIGURE 3 is a block diagram of an alternative 
embodiment of the digital video processing unit of 
FIGURE 1. 

DETAILED DESCRIPTION OF THE INVENTION 

FIGURE 1 illustrates an all-digital image display 
system 10 in accordance with the invention. Display 
system 10 comprises a signal interface 11, a video 
processing unit 13. a picture quality unit 14. a frame 
buffer 15. a spatial light modulator (SLM) 16. a color 
wheel 17, a display optics unit 18, and a timing unit 19. 
FIGURE 1 illustrates only those components of a dis- 
play system significant to main-screen video processing 
and display. Other components, such as might be used 
for processing synchronization signals, audio signals, 
secondary screen features such as dosed captioning, 
or graphics displays, are not shown. The following 
description is primarily directed to video processing unit 
13. 

For purposes of this description, system 10 has a 
DMD-type SLM 16. Comprehensive descriptions of 
DMD-based display systems, without the features of the 
present invention, are set out in U.S. Patent No. 
5,079,544, entitled "Standard IndeperdentJDjgitized 
r5~b&Q^r 6eo SvsterTr ' US " Patent Serial No L08/147,2 ^.9r^nti- 
'led "Digital Television System", and in U.S. Patent No. 
5,452,024, entitled "DMD Display System", each 
assigned to Texas Instruments Incorporated System 10 
could be used with ether types of SLMs. such as LCDs. 



Signal interface 11 receives digital input signals, 
such as the digital bitstreams provided by proposed 
advanced television (ATV) standards. For purposes of 
this description, it is assumed that the input signal is a 
5 compressed data signal. For purposes of example, it is 
assumed that the data is compressed in accordance 
with the MPEG standard and contains the data and tim- 
ing information consistent with that standard. In accord- 
ance with the MPEG standard, the data is in YCbCr ■ 
w (color difference) format. However, the invention could 
be useful for systems that receive data compressed by 
other methods. The input signal into signal interface 1 1 
might come from any one of a variety of sources, such 
as a television tuner, a data memory device, or via a 
is computer network. 

Signal interface 11 demultiplexes and buffers the 
input signal, so as to provide compressed video data as 
well as audio and timing signals. Signal interface 1 1 has 
appropriate filters and memory for performing these 
20 tasks. For example, an MPEG bitstream may require 
transport layer demultiplexing. 

Video processing unit 13 receives compressed 
video data from signal interface 1 1. As explained below 
in connection with FIGURE 2, video processing unit 13 
25 decompresses the data and scales (re-sizes) the result- 
ing image to match the display resolution of SLM 16. 

Picture quality unit (PQU) 14 receives the decom- 
pressed and scaled video data from video processing 
Unit 13. This data may be referred to as "YCbCr pixel 
30 ; data". PQU 14 performs tasks such as colorspace con- 
versipn and "de-gamma. Color space conversion con- 
verts YCbCr data to RGB data. The data may also 
undergo a de-gamma process, which linearizes the 
RGB data by removing the effect of a gamma correction 
35 that is processed into video signals at . transmission. 
PQU 14 may also perform error diffusion to reduce 
quantization artifacts. 

Frame buffer 15 receives processed pjxjgl^ajajrom 
PQU 14. It formats the data into "bit-plane" format, and 
40 delivers the bit-planes to SLM 16 one at a time. As dis- 
cussed in the Background, the bit-plane format permits 
each display element of SLM 16 to be turned "ON" or 
"OFF" in response to the value of one bit of data at a 
time. The formatting is performed by hardware associ- 
45 ated with frame buffer 1 5. 

In a typical display system 10. frame buffer 15 has 
a "double buffer" memory, which means that it has a 
capacity for at least two display frames. The buffer for 
one display frame can be read out to SLM 16 while the 
so buffer for another display frame is being written. The two 
buffers are controlled in a "ping-pong" manner so that 
data is continuously available to SLM 16. 

The bit-plane data from frame buffer 15 is delivered 
to SLM 16. Details of a suitable DMD-type SLM 16 are 
55 set out in US. Patent No. 4,956,619, entitled "Spatial 
Light Modulator", which is assigned to Texas Instru- 
ments Incorporated. Essentially. SLM 16 uses the data 
from bit-ptane buffer 15 to address each display ele- 
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ment of its display element array. The "ON" or "OFF" 
state of each display element forms an image. During a 
frame period. SLM 16 generates an image for each of 
three different colors (red. green, and blue) of a picture. 
These three images are sequentially displayed through 
a color wheel 17. The viewer's eyes integrate the 
sequential images so that the viewer perceives a full- 
color picture. 

Display optics unit 18 has optical components for 
illuminating SLM 16, for receiving the image from SLM 
16. and for projecting the image to a screen or other 
image plane. These components include a light source, 
illumination path optics, and a projection lens. 

in other embodiments, three SLMs could be used to 
concurrently display images of three colors, such as 
red. green, and blue. These images would be optically 
combined and perceived by the viewer as a full-color 
image. Still other embodiments might use two SLMs 
with images of different colors being either sequential or 
combined. 

Master timing unit 19 provides various system con- 
trol functions. Timing unit 19 may be implemented with 
a field programmable gate array (FPGA), to handle dif- 
ferent frame resolutions and frame rates. 

FIGURE 2 is block diagram of digital video process- 
ing unit 13. As explained below, video processing unit 
13 combines decompression and image resizing opera- 
tions. Separate processing "engines" 24 and 25 perform 
these operations, first decompression and then resiz- 
ing. These engines 24 and 25 share a memory 23 that 
stores the data to be operated on. A request-driven 
memory manager 22 handles the tasks of supplying the 
proper data at the proper time to each engine 24 and 
25. 

A feature of the invention is that the components of 
video processing unit 13 are designed so that they can 
be fabricated with integrated circuits. For example, the 
processing engines 24 and 25 and the memory man- 
ager 23 might be fabricated as one integrated circuit. 
This type of circuit is often referred to as an ASIC (appli- 
cation specific integrated circuit), with the applications 
being decompression and re-sizing of digital image 
data. As another example, memory 23 might also be 
"on-chip", such that the entire video processing unit 13 
is a single integrated circuit. 

Control interface 21 may be included to handle var- 
ious control signals. For example, control interface 21 
might receive and handle signals that support on- 
screen displays or override scaling for a picture-in-pic- 
ture. If included, control interface 21 would typically 
interface to a main system processor of system 10. 

Memory manager 22 receives the compressed 
video data and stores it in memory 23. Memory man- 
ager 22 handles the task of providing the appropriate 
data at the appropriate times to decoding engine 24 and 
scaling engine 25. The specific tasks of memory man- 
ager 22 are to: extract compressed data from interface 
1 1 and place it in memory 23, move compressed data 



from memory 23 to decoding engine 24, move decom- 
pressed data from decoding engine 24 back into mem- 
ory 23, move decompressed data from memory 23 to 
scaling engine 25, and arbitrate any conflicting requests 
5 for data from the decoding engine 24 and the scaling 
engine 25. 

Memory manager 22 is request-driven, in that data 
is delivered to decoding engine 24 and scaling engine 
25 in response to requests for data from these engines. 
w Memory manager 22 may be implemented as a hard- 
coded micro-controller. 

Memory 23 may be comprised of conventional 
dynamic random access memory devices (DRAMs). 
The size of memory 23 is related to various factors. 
is such as the image resolution and the compression ratio. 
Typically, memory 23 is sufficiently large to permit simul- 
taneous storage of at least part of a frame of com- 
pressed data being read in. a frame being processed, 
and at least part of a frame of decompressed data being 
20 read out. A typical size of memory 23 might be 1 6 meg- 
abits for a standard definition signal. 

A memory bus 22a between memory manager 22 
and memory 23 carries data, addresses, and the control 
signals necessary for reading and writing memory 23. 
25 The control signals include a write enable and column 
and row address strobes. 

Decoding engine 24 is a processor programmed to 
decompress the video data. It may be programmed to 
support various compression standards, such as the 
30 JPEG, MPEG, MPEG2, Px64. CCITT, etc. The pro- 
gramming can be for either still or full-motion decom- 
pression algorithms. Decoding engine 24 could be a 
multi-format decoding engine, switchable between 
decompression algorithms so as to perform whatever 
35 decompression method is appropriate for the input sig- 
nal. The output of decoding engine 24 is decompressed 
pixel data, which in the example of this description, is in 
YCbCr format. , 

An example of a suitable decoding engine 24 is the 
40 decoding engine of the CL450 MPEG Video Decoder 
manufactured by C-Cube Microsystems. It provides 
decompressed YCrCb data with an image size consist- 
ent with the input signal. 

A decoding engine bus 22b carries control informa- 
45 tion and data between decoding engine 24 and memory 
manager 22. The control information includes requests 
for data from decoding engine 24, which requests data 
as needed for processing. In response to a request, 
memory manager 22 generates the address of the next 
so data to be decoded, retrieves the data from memory 23, 
and delivers it to decoding engine 24. Bus 22b is bidi- 
rectional - after decompression, decoding engine 24 
delivers the data back to memory manager 22, which 
generates an address for storing it in memory 23. 
55 Scaling engine 25 operates on the decompressed 
video data. Essentially, scaling engine 25 is a processor 
programmed to scale the image so as to resize the 
image to the display size (resolution) of SLM 16. As 
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stated in the Background, SLM 1 6 does not always have 
a display resolution that matches the resolution of the 
input signal. For example, a typical SLM 16 might have 
a display resolution of 1280 x 1024, whereas, for HDTV, 
full transmission resolution is 1920 x 1080 pixels. Seal- s 
ing engine 25 upscales or downscales the input signal 
to fit the SLM size in either or both dimensions. Horizon- 
tal scaling changes the number of active pixels per line 
and vertical scaling changes the number of active lines 
per frame. Various scaling algorithms are known in the w 
art of image processing, such as those that use interpo- 
lation or FIR filtering. 

As an example of the scaling process, the image 
represented by the decompressed data from decoding 
engine 24 might be smaller than the display size of SLM is 
16, both vertically and horizontally. For vertical scaling, 
scaling engine 25 would generate new lines of data by 
filtering neighboring lines of the existing image, request- 
ing these neighboring lines as needed. It may need the 
same line more than once during this process. Likewise, 20 
for horizontal scaling, scaling engine 25 would generate 
new pixels by filtering neighboring pixels of the existing 
image, requesting them from memory manager 22 as 
needed. 

Scaling engine 25 is provided with input and output 25 
image size parameters. The output size of the image is 
determined by the display size of SLM 16. The input 
image size is provided by decoding engine 24, which 
decodes this data from the compressed video data 
stream. 30 

A scaling engine bus 22c carries control signals 
and data between scaling engine 25 and memory man- 
ager 22. The control signals include requests for data as 
needed for processing. After scaling, the video data is 
delivered to PQU 14. 35 

An example of a suitable scaling engine 25 is the 
scaling engine of the Acuity Resizer device, manufac- 
tured by Genesis Microchip, Inc. The Acuity gm833x2 
device operates on YCbCr data, using FIR filters on two 
separate channels. 40 

FIGURE 3 illustrates an alternative embodiment of 
video processing unit 13. A feature of this embodiment 
is that memory 33 has two data output ports and deliv- 
ers data directly to decoding engine 24 or to scaling 
engine 25. The dual output ports can be used to meet 45 
high bandwidth demands. Like the embodiment of FIG- 
URE 2, the data is delivered in response to requests 
delivered to memory manager 32. The memory bus 32a 
carries a signal that indicates the destination of the 
requested data. 50 

Like the embodiment of FIGURE 2, the video 
processing unit 13 of FIGURE 3 may be easily fabri- 
cated as two integrated circuits, one tor decoding 
engine 24, scaling engine 25, and memory manager 32, 
and the other for memory 33. Alternatively, video 55 
processing unit 13 could be fabricated as a single inte- 
grated circuit. 

Although the present invention has been described 



in detail, it should be understood that various changes, 
substitutions, and alterations can be made hereto with- 
out departing from the spirit and scope of the invention. 

Claims 

1 . A digital video processing unit comprising: 

a decoding engine for decompressing com- 
pressed video data received on an input signal, 
thereby providing decompressed video data; 
a scaling engine for scaling said decom- 
pressed video data; 

a memory for storing said compressed video 
data prior to delivery to said decoding engine, 
and for storing said decompressed video data 
prior to delivery to said scaling engine; and 
a memory manager in data communication 
with said decoding engine, said scaling engine, 
and said memory, said memory manager being 
operable to respond to data request signals 
from said decoding engine and from said scal- 
ing engine, to generate address signals for 
storing data in said memory. 

2. The video processing unit of Claim 1, wherein said 
decoding engine, said scaling engine, and said 
memory manager are formed on a single integrated 
circuit 

3. The video processing unit of Claim 1 or Claim 2 
wherein said video processing unit is formed on a 
single integrated circuit. 

4. The video processing unit of any preceding claim, 
wherein said scaling engine is operable for resizing 
an image represented by said video data. 

5. The video processing unit of any preceding Claim, 
wherein said memory has a first output port for 
delivering said compressed video data to said 
decoding engine and a second output port for deliv- 
ering said decompressed video data to said scaling 
engine. 

6. The video processing unit of any preceding Claim, 
wherein said memory has a single data port for 
delivering said compressed video data and said 
decompressed video data to said memory man- 
ager. 

7. The video processing unit of any preceding Claim, 
wherein said compressed video data is com- 
pressed in accordance with the MPEG standard 
and wherein said decoding engine is programmed 
to perform MPEG decompression. 

8. The video processing unit of any preceding Claim, 
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wherein said compressed video data is in YCbCr 
format. 

o -me video processing unit of any preceding Claim 
9 - wherein *d decoding engine is JJ^J-f* 
decoding input image size data from said ram 
orSed video data and is in data commun.cat,on 
STSd scaling engine for delrvering sa,d mput 
image size data to said scaling engine. 
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10. A video display system comprising: 

a signal interface for receiving a compressed 

^TdeTprLssing un« having a decoding 
engine for decompressing said compressed 
video data thereby providing decompressed 
video data, a scaling engine for resizing sad 
decompressed video data to match a delivered 
ds^size thereby providing a first forma 
S data, and a shared memory access.ble by 
said decoding engine and sad scaling engine; 
a picture qualrty unit operable to convert said 
M Smat pixel data to a second format pixel 

atametuffer operable to ^ 
format pixel data into bit-plane format and to 
deliver said second format pixeldata ^to j 
□lay said display being operable to display 
Sges based on said second format pixel data 

1 1 The video display system of Claim 10. wherein said 
11 " com?isi a spatial light modulator opera- 

ble to display images based on said second format 

pixel data. 

12. The video display of Claim 10 or Cairn 1 ^wherein 
said first format pixel data comprises YcbCr pixel 
data. 

13 The video display of any of Claims 10 to 12. 
wherein said second format pixel data composes 
RGB pixel data. 

14. The video display system o. any of Claims^ io » to 13. 
wherein said decoding engine and said scaling 
e5e are formed on a single integrated arc* 

1 5 . The vdeo display system of any of C lai ms 1 CT IdH 
wherein sad video process.ng unit is formed as a 
single integrated circuit. 

16 The video display system of any of Claims 10 to 15 
whenJn said Vemory has a first outout port tor 
deSg said compressed video to said decoding 
enS and a second output port for delivering sa,d 
decompressed video data to said scaling engme. 



1 7. Tne video display system of any of Claims 1 0 to 16 
wherein said video processing unrt further com 
prises a request-driven memory manager. 

5 iamevideod.splaysystemofClaim17.whereinsa« 
first processor. sad second processor, and sad 
memory manager are formed on asmgle integrated 
circuit 

» 19 The video display system of Claim .17 or Claim 18, 
W wherein said memory has a singie data .port tor 
delivering said compressed video and sad decom- 
pressed video data to said memory manager. 

20 me video display system of any of Claims 10 to 19. 
wherein sad compressed video data « com- 
^ed in accordance with the MPEG standard 
and wherein said decoding engine is programmed 
to perform MPEG decompression. 
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21 A method of providing video data tor use in a video 
display system, comprising the steps ot. 

receiving an input signal carrying compressed 
video data: 

storing said data in a video process.ng mem- 

dSompressing said compressed ^ 
said compressed video data being de*ve ed 
from said video processing memory, thereby 
providing decompressed video data; 
storing said decompressed data in said vdeo- 
processing memory; and 
scaling said decompressed vdeo data to 
match a determined display size, sad decom 
pressed video data being delivered from sad 
vdeo processing memory. 

22 me method of Claim 21 , wherein said step of s oal- 
ing said decompressed video data compnsesre^z. 

ina said decompressed video data such that sad 
imagt represent by said video data substantially 
corresponds to said display size. 

Tine method of Claim 21 or Claim 22. further com- 
bing provding a spatial light modulator for form- 
ing a display. 

me method of any of Claims 21 further com- 
prising delivering said compressed vdeo data and 
X decompressed video data in response to 
Quests foVdata during sad decornpressmg and 
scaling steps. 

25 me method of any of Claims 21 to 24. wherein said 
M - jessing sW comprises generating .n P u 
imagesize data and performing sad scaling step .s 
by using said input image. 
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